Contour Based Automatic Scene Segmentation in Image Sequences

Scene segmentation is a challenging problem that can be a prelude to many further applications in computer vision. This paper describes a novel MAP framework for automatic and robust scene segmentation in image sequences under mobile camera without any human interaction. First, an initial partition of the scene is achieved by watershed segmentation algorithm. However segmentation based on single frame often suffers from over-segmentation and simplified-segmentation problem due to the absence of 3D information. To eliminate these problems, we propose to analyze the 2D motions on the two sides of the segmentation contours to judge whether a contour is the ”space contour” or a ”texture contour”, thus eliminate over-segmentations and simplified-segmentations. To achieve the best segmentation, we use multi key frames of image sequences to seek for the global optimal scene segmentation by employing a MAP estimator. We demonstrate high accuracy and robustness of our algorithm by experiments on real video sequences.